Skip to content

fix(#10): post-merge docs/repro fixup — addresses 5 reviewer items#18

Open
hitome0123 wants to merge 1 commit into
billion-token-one-task:mainfrom
hitome0123:fix/issue-9-docs-repro-fixup
Open

fix(#10): post-merge docs/repro fixup — addresses 5 reviewer items#18
hitome0123 wants to merge 1 commit into
billion-token-one-task:mainfrom
hitome0123:fix/issue-9-docs-repro-fixup

Conversation

@hitome0123
Copy link
Copy Markdown

Summary

Follow-up to merged PR #10. Addresses all 5 cleanup items from Protocol-zero-0's post-merge review (2026-05-16) plus a couple of adversarial nits we caught on second pass.

Reviewer's note: "第 5 条做完,前 4 条会顺带暴露并修掉" — building verify_acceptance.sh indeed surfaced the stale commit fields and the non-deterministic delta, so this PR closes everything in one pass.

Reviewer checklist

  • 实现科学机会评估排序与自动实验闭环模块 #1 missing scripts/demo_agenda_loop.py — new thin wrapper that delegates to build_agenda_loop_acceptance.main(). Inserts repo root onto sys.path so the documented one-liner python scripts/demo_agenda_loop.py works directly (not just with -m).
  • Add opportunity triage queue #2 stale commit: ef15797 in all 6 acceptance bundles — re-ran every builder; all 6 now carry commit: 44ae888a0c1c…. Added scripts/build_manuscript_venue_routing_umbrella.py so the umbrella JSON (issue [Epic] Manuscript Venue Routing + Multi-Template Pipeline (会议路由 + 多模板论文管线) #11) is regenerated deterministically from sub-bundles + git rev-parse HEAD instead of being hand-written.
  • fix: Debug startup resilience #3 false byte-reproducibility claim on result_packet_sha256 — added _strip_non_deterministic() covering wallclock-derived metrics (latency_ms_*, peak_memory_mb, latency_speedup_x), DB auto-increment ids, and abs paths. Renamed the JSON field to result_packet_structural_sha256 so the claim is honest. Also applied the same strip to experiment.delta so the bundle JSON itself is byte-stable across runs (was still leaking latency_speedup_x after the rename — caught on second pass).
  • [Feature Request] Closing the loop: Integrating Automated Manuscript Drafting, AI Peer-Review, and Publication Pipeline #4 agents/venues.yamlmanuscript_venues/venues_v1.yaml + stale venue names — fixed every reference in README.md, docs/top_venue_manuscript_chain.md, and the diagram block. The 6 actually registered venues are now consistently named everywhere: iclr2026, neurips2024, icml2024, acl_arr, cvpr2024, arxiv_plain. `grep -rn 'agents/venues.yaml'` returns 0 hits; `grep -rn 'neurips2025|emnlp2025|iccv2025'` returns 0 hits.
  • Add SciForge benchmark pipeline and safe RL review pass #5 scripts/verify_acceptance.sh — one-liner that any contributor can run from a clean venv. Wipes scratch DBs, runs all 5 builders + the umbrella, parses each artifact JSON, cross-checks the commit field against git rev-parse HEAD, and prints PASS — 6/6 acceptance bundles regenerated at commit <sha> only when everything succeeds. Uses set -Eeuo pipefail so any builder failure short-circuits with a clear message.

Verification (re-run on the head commit of this PR)

```
$ bash scripts/verify_acceptance.sh
[verify] regenerating d1 template router (python -m scripts.build_d1_acceptance)
[verify] regenerating d2 top-venue adapters (python -m scripts.build_d2_acceptance)
[verify] regenerating d3 format linter (python -m scripts.build_d3_acceptance)
[verify] regenerating d4 manuscript routing (python -m scripts.build_d4_acceptance)
[verify] regenerating agenda loop (#9) (python -m scripts.build_agenda_loop_acceptance)
[verify] regenerating umbrella (#11) (python -m scripts.build_manuscript_venue_routing_umbrella)
[verify] verifying 6 acceptance bundles exist + parse + match HEAD
ok artifacts/agenda_loop_acceptance.json (commit=44ae888a0c1c)
ok artifacts/d1_template_router_acceptance.json (commit=44ae888a0c1c)
ok artifacts/d2_top_venue_adapters_acceptance.json (commit=44ae888a0c1c)
ok artifacts/d3_format_linter_acceptance.json (commit=44ae888a0c1c)
ok artifacts/d4_manuscript_routing_api_acceptance.json (commit=44ae888a0c1c)
ok artifacts/manuscript_venue_routing_acceptance.json (commit=44ae888a0c1c)

================================================================
PASS — 6/6 acceptance bundles regenerated at commit 44ae888

```

Epic-scope tests (touched paths only):

```
$ python3 -m pytest tests/test_top_venue_adapters.py tests/test_venue_router.py \
tests/test_format_linter.py tests/test_manuscript_routes.py \
tests/test_template_adapter.py tests/test_venue_router_tiebreak.py -q
50 passed, 6 subtests passed in 0.65s
```

Reproducibility proof (two consecutive runs of the agenda loop builder):

```
run1.experiment.delta = {"approximation_l2_distance": 10.13387978273153, "relative_error": 0.7670742721257483}
run2.experiment.delta = {"approximation_l2_distance": 10.13387978273153, "relative_error": 0.7670742721257483}
run1.result_packet_structural_sha256 = 087c1d13348aefbad55a921f3380984de3719cfc581ff300fbe6adf6ab6f67e8
run2.result_packet_structural_sha256 = 087c1d13348aefbad55a921f3380984de3719cfc581ff300fbe6adf6ab6f67e8
```

Documented one-liner now works directly (no -m needed):

```
$ python scripts/demo_agenda_loop.py
[ok] wrote artifacts/agenda_loop_acceptance.json
selection_id=1 run_id=2 gate=block bundle_id=None review_id=1 plan_id=1
```

Files

New (3)

Modified (11)

Test plan

  • `bash scripts/verify_acceptance.sh` → PASS
  • Epic-scope pytest suite (6 files) → 50 passed
  • `python scripts/demo_agenda_loop.py` (no `-m`) → exit 0
  • Two-run reproducibility diff on `experiment.delta` + `result_packet_structural_sha256` → byte-identical
  • `grep -rn 'agents/venues.yaml' .` → 0 hits
  • `grep -rn 'neurips2025|emnlp2025|iccv2025' .` → 0 hits

Related: #9, #10, #11, #12, #13, #14, #15.

🤖 Generated with Claude Code

…es 5 reviewer items

PR billion-token-one-task#10 was merged but the post-merge review (Protocol-zero-0, 2026-05-16)
flagged 5 cleanup items. This commit closes all 5, plus a couple of
adversarial-review nits surfaced by re-running the recipe ourselves.

1. Missing scripts/demo_agenda_loop.py referenced in the PR repro recipe.
   Added a thin wrapper that delegates to build_agenda_loop_acceptance.main();
   inserts repo root onto sys.path so the documented one-liner
   `python scripts/demo_agenda_loop.py` works (without -m).

2. Stale `commit: ef15797` in all 6 acceptance bundles vs HEAD 44ae888.
   Re-ran every builder; added scripts/build_manuscript_venue_routing_umbrella.py
   so the umbrella JSON (issue billion-token-one-task#11) is regenerated deterministically
   from sub-bundles + git HEAD instead of being hand-written.

3. `agenda_loop_acceptance.json` claimed byte-reproducible but the
   underlying packet contained wallclock-derived fields. Added
   _strip_non_deterministic() that drops latency_ms_*, latency_speedup_x,
   peak_memory_mb, DB ids, and abs paths before hashing; renamed the JSON
   field result_packet_sha256 -> result_packet_structural_sha256 so the
   claim is honest. Also applied the same strip to `experiment.delta`
   so the bundle JSON itself is byte-stable across runs.

4. README + docs referenced agents/venues.yaml (wrong path) and stale
   venue names neurips2025/emnlp2025/iccv2025. Fixed to
   manuscript_venues/venues_v1.yaml and the actual 6 registered venues:
   iclr2026, neurips2024, icml2024, acl_arr, cvpr2024, arxiv_plain.

5. New scripts/verify_acceptance.sh — one-liner that wipes scratch DBs,
   runs all 5 builders + the umbrella, parses each artifact JSON,
   cross-checks the commit field against `git rev-parse HEAD`, and
   prints `PASS — 6/6 acceptance bundles regenerated at commit <sha>`
   only when everything succeeds.

Verification:

  $ bash scripts/verify_acceptance.sh
  PASS — 6/6 acceptance bundles regenerated at commit 44ae888

  $ python3 -m pytest tests/test_top_venue_adapters.py \
      tests/test_venue_router.py tests/test_format_linter.py \
      tests/test_manuscript_routes.py tests/test_template_adapter.py \
      tests/test_venue_router_tiebreak.py -q
  50 passed, 6 subtests passed in 0.65s

  $ python scripts/demo_agenda_loop.py   # directly, no -m
  [ok] wrote artifacts/agenda_loop_acceptance.json

Two runs of build_agenda_loop_acceptance now produce identical
result_packet_structural_sha256 (087c1d13...) and identical
experiment.delta {approximation_l2_distance, relative_error}.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@cla-assistant
Copy link
Copy Markdown

cla-assistant Bot commented May 17, 2026

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

1 similar comment
@cla-assistant
Copy link
Copy Markdown

cla-assistant Bot commented May 17, 2026

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants